December 15, 2023
By Bob O'Donnell
In an era in which it’s clear that the PC still matters, Intel is making a statement about where it sees the category heading with the long-awaited launch of its Core Ultra chip (codenamed “Meteor Lake”). Not surprisingly, the direction is straight towards AI.
At the company’s AI Everywhere event in NYC, Intel debuted both the Core Ultra for notebook PCs and its 5th Generation Xeon CPU for servers and data centers. Both chips offer accelerated AI capabilities and important improvements in power efficiency and performance.
Core Ultra combines several critical new technologies and capabilities into a single SOC (System on Chip)—in fact, Intel claims it’s their biggest architectural leap in PC processors in about 40 years. The one change likely to get the most attention is the company’s first ever integration of a new component called the NPU, or Neural Processing Unit, into an SOC design. The NPU is specifically tasked with accelerating certain types of AI workloads, freeing the CPU and GPU (also built into the chip) to do other tasks. In the process, it enables new types of applications to run on PCs and improves the laptop’s battery life and overall efficiency.
That’s a pretty heady combination, and Intel is using it to herald the dawn of the AI PC era—perfectly timed, of course, to tap into the huge focus on generative AI that’s swept over the tech industry and much of the world this past year. To be fair, AMD actually had the first PC chip with a built-in NPU earlier in the year with its Ryzen 7040 and just unveiled the updated Ryzen 8040 with a second generation NPU last week (see “AMD Makes Definitive GenAI Statement” for more). Given Intel’s massive position in PCs (as well as the limited software support for AMD’s initial NPU), however, there’s little doubt that the first Core Ultra-based PCs—which are available as of today—will be seen by many as the first in this burgeoning new category.
From a practical perspective, most people will appreciate the more traditional enhancements Intel has made to the chip—even though they may not be as sexy as the NPU. Core Ultra features not only the combination of up to 6 performance P Cores and 8 efficiency-focused E cores, but it also includes two new ultra low-power cores they’re calling a low-power island that can kick in when system demands are low (such as when the PC isn’t doing much) and extend battery life even further. The new GPU in Core Ultra is based on the company’s standalone Arc chip, offers up to 8 Xe graphics cores and features an impressive 2x improvement versus previous Intel integrated graphics solutions.
Core Ultra is interesting at a manufacturing level as well. The main CPU element is the first chip Intel is building with its Intel 4 process technology (the second of 5 manufacturing nodes the company is on track to complete over the next four years), and it includes an advanced version of the Foveros 3D chip-stacking technology. Foveros allows Intel to combine multiple different chiplets made at multiple different process nodes into a single, more sophisticated SOC design.
One of the other things that Intel is doing with Core Ultra as part of its overall AI PC push is talking about how all three of its main computing elements—the NPU, CPU and GPU—all work together on AI-related workloads. Most of the industry’s focus to date has been on the TOPS (Tera Operations Per Second) of the NPU, with Qualcomm in particular highlighting how its forthcoming Snapdragon X Elite will have an NPU with 45 TOPS. The raw TOPS performance of Intel’s NPU in Meteor Lake is 10 and the recently updated one in AMD’s Ryzen 8040 is supposed to be 16. Not surprisingly, Intel and AMD are both focusing on total system TOPS, because it turns out both the CPU and GPU can have a big impact on those numbers. Intel is claiming up to 34 system TOPS with Core Ultra and AMD up to 39 system TOPS with the 8040 (Qualcomm said theirs will have a system TOPS of up to 75).
In truth, there’s an enormous amount of industry debate about the real value of TOPS numbers of any kind, but particularly for the NPU. To Intel and AMD’s credit, in practical use, AI-focused applications are using the CPU and the GPU already and will be adding support for NPUs over time. In other words, there aren’t a whole lot of applications that leverage only the NPU, so the current real-world value of TOPS performance benchmarks is up to question. This is particularly true because so much of the AI-related work being done on PCs right now is done via the cloud and doesn’t do any local processing (think ChatGPT, Microsoft 365, etc.). Eventually, of course, the hope is that more of this work will start to be done locally, but as with most software-related efforts, it will take time.
In the meantime, there are a few examples of applications that can start to leverage the hardware that’s available on Core Ultra, with Intel showing off several interesting demos. Most notable was Superpower (the new name for the previously demo’d Rewind.ai), which can record all the things you do and see on your PC and then easily find them and allow you to generate new content (such as emails and documents) based on the documents and data you have on your own PC. At present, the company is using a 7B parameter version of Meta’s Llama2 foundation model at its engine, but discussions with the company’s founders suggest there could be other possibilities in the future. From an operational perspective, what’s interesting about Superpower is that it uses both the onboard NPU and CPU on Core Ultra, so it represents a good real-world example of how AI-focused apps are likely to evolve. Unlike the Mac version of the app, SuperPower on Core Ultra equipped PCs can run even when it isn’t connected by leveraging the local LLM and the NPU. Macs don’t currently have NPUs, so it requires an internet connection to function. Another demo Intel created in conjunction with HP involved a gaming app that used the NPU to do live streaming, freeing up the entire GPU for the game.
While these are definitely interesting, the obvious question that many people will still have is what does this mean to the average business PC user or consumer? Intel’s EVP and GM of its client computing group, Michelle Johnston Holthaus, provided a refreshingly honest answer during a Q&A at the event: it’s still early days, and they don’t know for sure. There’s little doubt that AI-enabled PCs are going to represent an important new era for the devices in the future. It’s clear, however, that people are still figuring out what exactly that’s going to be, so it’s great to see Intel acknowledge that.
As the long-time industry leader in PC architectures, Intel’s endeavors in driving forward the idea of an AI-capable PC are definitely important and newsworthy. Between the company’s enormous marketing efforts, its work with over 100 software developers to bring more AI-accelerated applications to market, and the huge infrastructure and ecosystem it has built over the past few decades, Intel can (and likely will) have an outsized influence on how the idea of AI-powered PCs come to market. Given what a critical transition this will likely prove to be, it’s great to see the company taking these important first steps.
Here's a link to the original column: https://www.linkedin.com/pulse/intel-refines-its-computing-vision-bob-o-donnell-8vymc/?trackingId=qNlV1CZBTcS%2Bfk9Gzl%2FMxQ%3D%3D Bob O’Donnell is the president and chief analyst of TECHnalysis Research, LLC a market research firm that provides strategic consulting and market research services to the technology industry and professional financial community. You can follow him on LinkedIn at Bob O’Donnell or on Twitter @bobodtech. |
